Skip to content

openkal c-environment: withdraw the _WIN32 adaptations, add posix/platform labels (draft, blocked on two releases) - #439

Draft
Sunrisepeak wants to merge 15 commits into
mainfrom
openkal-c-environment
Draft

Sunrisepeak wants to merge 15 commits into
mainfrom
openkal-c-environment

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

What this is

The mcpp-index leg of the openkal C-environment wave (design:
openkal/.agents/docs/2026-09-18-openkal-c-environment-and-personalities-design.md,
plan: the sibling 2026-09-18-c-environment-execution-plan.md). Draft,
opened now so CI can react to the comments and mechanism changes below
before the two releases this PR depends on exist — not to be merged yet.

This PR cannot merge before openkal-musl 0.15.0 and
openkal-llvm-runtime 0.11.0 are published
, and per index.toml's own
rule, not before min_mcpp and the CI MCPP_VERSION pin can move together
to the mcpp release that understands [c-abi]. Landing order, from the
plan's §5 (steps 1–3 happen elsewhere, in mcpp and the two openkal repos;
steps 4–6 are this repo and after it):

  1. mcpp #668 merges and releases ([c-abi] must exist in a published engine)
  2. that mcpp release enters xim-pkgindex
  3. openkal-musl 0.15.0 and openkal-llvm-runtime 0.11.0 release (GitHub tag + GitCode mirror)
  4. this PR: raise min_mcpp/MCPP_VERSION together, register the two new versions, withdraw the zlib/mbedtls -U_WIN32 adaptations, re-measure — all in one PR, because the gate and the descriptors it gates must take effect together or an older engine misreads them silently
  5. a separate openkal-repo PR for the README/doc wording (no SPEC change, no release needed)
  6. sandbox verification, once the index side is actually published

What is here now, and what is still blocked

Done, verified against the currently-published mcpp/openkal-musl (no
[c-abi] yet, so this only covers what does not depend on it):

  • Withdrew compat.zlib.lua's and compat.mbedtls.lua's target_cfg
    scoped to cfg(all(windows, c-abi = "musl")) — hand-written
    -U_WIN32/-D__unix__ predecessors of what [c-abi] presents = "posix"
    will do for the whole target once it exists. mbedtls's block is gone
    outright; zlib's -include unistd.h half is replaced by hoisting the
    existing mcpp_zlib_config.h include (already used on Linux/macOS) to
    apply unconditionally, since the header's own #if !defined(_WIN32) guard
    is the real platform test. A follow-up commit records that this flag is
    package-private (verified against compile_commands.json) and that its
    safety today rests on every target this index measures being LP64 — noted
    in the descriptor for whoever adds an ILP32 target next.
  • Checked the rest of pkgs/c/*.lua for the same shape: no other instance.
    compat.libarchive.lua's two target_cfg entries are a different
    concern (kernel/SDK header reachability) and are untouched.
  • Added a second, orthogonal kind label (posix / platform) to
    tests/openkal/compat.py and wired it through .xpkgindex/plugins/mcpp.py
    the same way the existing openkal tag/facet is wired — derived from
    measurement and declaration only, never hand-assigned. native is
    explicitly deferred (one sentence, no stub). The package-level badge is
    documented as a summary of the strictest measured target; the per-target
    table is the stated authority.

Cannot be done yet, and is marked rather than faked:

  • pkgs/o/openkal-musl.lua (0.15.0) and pkgs/o/openkal-llvm-runtime.lua
    (0.11.0): no version entry, no sha256 — neither exists. Each carries a
    comment next to its version table naming what the pending version changes.
  • index.toml: no min_mcpp/latest_mcpp bump — no mcpp release understands
    [c-abi] yet. A dated comment records the landing order above.
  • The withdrawn zlib/mbedtls adaptations are verified against today's
    mcpp/openkal-musl (native Linux build/test still pass, cflags unchanged
    there) but the actual openkal-Windows + presents = "posix" path cannot be
    built or measured here — nothing exists yet to build it against.
  • .xpkgindex/openkal-compat.json is untouched. It has no kind data yet;
    the next real tests/openkal/compat.py run, once there is something new
    to measure, populates it.

Verification run

  • mcpp xpkg parse + this repo's lua lints, over every touched descriptor
    and a full pkgs/*/*.lua sweep
  • mcpp test for the zlib and core (mbedtls) workspace members, native
    Linux
  • xpkgindex generate . --output ... --offline --strict (the command
    site-check.yml runs): clean, no warnings, all landing pages present
  • The new facet/badge/table column confirmed against a throwaway copy of
    .xpkgindex/openkal-compat.json patched with synthetic kind values,
    then discarded — not committed, since inventing measurement data is
    exactly what this mechanism exists to avoid

…ilds against mcpp 2026.9.17.3

The two packages resolve openkal 0.13.0 from this index, so their releases
follow #437. mcpp 2026.9.17.3 carries #662: a dependency graph that supplies the
C library is no longer mixed with the host's own headers. validate.yml and the
openkal compatibility measurement both move to it, and the measurement takes
openkal-llvm-runtime 0.10.0 (openkal-musl 0.14.0, openkal 0.13).
libarchive reads __APPLE__ as the Apple SDK being present (TargetConditionals.h,
copyfile, quarantine) in four private units and in the generated configuration;
in an openkal graph for macOS the SDK is not there. __APPLE__ is removed from
libarchive's own units under cfg(all(macos, c-abi = "musl")); archive.h and
archive_entry.h do not test it. Measured with mcpp 2026.9.17.3: the archive
member cross-builds for aarch64-macos above openkal-macos 0.10.0.

Windows with musl is not adapted, and the reason is recorded here rather than
worked around: libarchive writes wide string literals, which the Windows target
makes sixteen bits while musl's wchar_t is thirty-two, and openkal-musl refuses
that mixture on purpose (musl/PATCHES.md). The measurement reports it.
The measurement against mcpp 2026.9.17.3 showed the archive member failing on
Linux above openkal: the Linux host configuration names linux/fs.h and its
neighbours, which belong to the kernel's interface and were reached only
because the compiler searched the host before mcpp#662. Under
cfg(all(linux, c-abi = "musl")) the generated configuration removes them as
it does on every other system; the member runs again.

The tinyhttps member names 0.3.1, which selects POSIX sockets on Windows with
musl; the measurement had been taken against 0.3.0.
A copy of a member is not a member of the workspace and did not inherit its
[indices], so compat resolved from the published index and a descriptor changed
in a pull request was not the one measured (the archive member measured
libarchive's published descriptor). The copy now names this checkout when the
member declares no [indices] of its own.
Measured by openkal-compat run 35222831479 with mcpp 2026.9.17.3,
openkal-llvm-runtime 0.10.0 and llvm@22.1.8, the host's mingw headers installed.
x86_64-linux-gnu: 27 members run, 3 fail (cmp-module and curl name Linux kernel
headers or epoll, which openkal does not provide; expat calls arc4random_buf,
which musl does not declare). x86_64-windows-gnu, run through Wine: 15 run,
15 fail, each on a Windows or MinGW header its package includes on _WIN32.
…i declaration replaces

Both descriptors carried a target_cfg scoped to cfg(all(windows, c-abi =
"musl")) that hand-corrected what the windows-gnu triple implied for the
library's own translation units: zlib's -U_WIN32 -include unistd.h, and
mbedtls's -U_WIN32 -D__unix__. Neither was wrong when it was written --
openkal-musl had no way yet to say what environment it presents, so the
descriptor said it instead, one package at a time.

openkal-musl now declares `[c-abi] presents = "posix"` (design:
openkal/.agents/docs/2026-09-18-openkal-c-environment-and-personalities-
design.md §3.2), and mcpp realises that declaration for the whole target
rather than per package. _WIN32 is simply absent and __unix__ is simply
present on openkal-Windows to begin with, so these two hand-rolled
predecessors of the mechanism are withdrawn rather than reproduced.

mbedtls's target_cfg block is removed outright: both of its flags were
purely about environment identity, exactly what presents = "posix" now
states once for the whole target.

zlib's case also needed something presents = "posix" does not itself
provide: mcpp_zlib_config.h (which defines Z_HAVE_UNISTD_H, and so pulls
in <unistd.h> for the read/write/close/lseek declarations gzlib.c and
friends call directly) was previously wired into the linux/macosx blocks
only, so windows-musl's target_cfg additionally hand-included
<unistd.h>. Since the header's own guard is `#if !defined(_WIN32)`, the
same platform test the cflags scoping used to perform by hand, hoisting
the -include to a single unconditional cflags entry reaches
openkal-Windows exactly as it already reaches Linux and macOS, and does
nothing on native (non-openkal) Windows where _WIN32 stays true -- no
target_cfg needed to say so. z_off_t now resolves to off_t rather than a
forced `long long` wherever the header's guard is open, but both are 8
bytes under the LP64 data model presents = "posix" also declares, and
tests/examples/zlib asserts that width dynamically via
zlibCompileFlags(), not a fixed type.

Checked the rest of pkgs/c/*.lua for the same shape (a target_cfg cflags
entry undoing a platform macro for windows+musl): no other instance.
compat.libarchive's two target_cfg entries (linux+musl dropping kernel
UAPI headers, macos+musl dropping __APPLE__) are a different concern --
kernel/SDK header reachability, not C-environment identity -- and are
untouched.

Verified: mcpp xpkg parse and the repo's lua lints on both files; `mcpp
test` for the zlib and core (mbedtls) members on this host still pass
(native Linux effective cflags are unchanged by this refactor, so both
came back from the build cache). The windows+musl path itself cannot be
built or measured here: it needs the mcpp release and openkal-musl
0.15.0 this same wave's index.toml and pkgs/o/openkal-musl.lua changes
mark as pending.
tests/openkal/compat.py already records, per member and target, whether
the build ran, built, or failed. This adds a second axis beside it,
answering a different question: not whether it worked, but what it
needed in order to.

    posix     built and ran using only the C environment the graph's C
              library presents
    platform  needs the platform's own interfaces

Both are derived, never hand-assigned: `kind_of()` reuses the one signal
already computed for this purpose, `platform_bound()` (a member
declaring a per-target dependencies table -- the same fact the site
already showed as "selects platform dependencies of its own"), together
with the measured `status`. platform_bound is "declares a platform
dependency"; docs/openkal-compat.md rule 1 already describes a feature's
`feature-deps` reaching a platform SDK shim as the way a descriptor
expresses "this adaptation reaches platform headers", so the one signal
covers both halves of the definition rather than needing a second,
separate heuristic that would have to guess at a package's internals.
`kind` is omitted, not guessed, when status is "fails".

A third label, `native` (built in a reduced ISO C form with no
POSIX-shaped package anywhere in the graph), is deliberately deferred:
that form does not exist yet (design:
openkal/.agents/docs/2026-09-18-openkal-c-environment-and-personalities-
design.md §7, §12 decision 5). It is not implemented here beyond the one
sentence saying so, in each of the three places this documents the
mechanism (compat.py's docstring, docs/openkal-compat.md, and the
plugin's OPENKAL_KINDS comment) -- no dict entry, no branch, no stub.

Wired into the site the same way the existing `openkal` tag and facet
are: `.xpkgindex/plugins/mcpp.py` gains OPENKAL_KINDS (paralleling
OPENKAL_LEVELS), an `_openkal_kind()` package-level reducer (paralleling
`_openkal_level()`), a `pkg.facets["openkal_kind"]` facet value, a
matching badge, and an `environment` column in the existing per-target
openkal table -- so the derivation is stated in the same place the
status is, both in the results file (same target dict) and on the site
(same table, same row).

Verified: `xpkgindex generate . --output ... --offline --strict` (the
same command .github/workflows/site-check.yml runs) builds clean, no
warnings, all landing pages present. Confirmed the new facet, badge and
table column actually render by patching a throwaway copy of
.xpkgindex/openkal-compat.json with synthetic `kind` values and
rebuilding -- not committed here, since inventing measurement data is
exactly what this mechanism is designed not to do. The checked-in
.xpkgindex/openkal-compat.json is unchanged; it was measured by the
previous mechanism and will pick up `kind` on the next real
tests/openkal/compat.py run.
…till needs

openkal-musl 0.15.0 and openkal-llvm-runtime 0.11.0 -- the versions that
declare and consume `[c-abi] presents = "posix"` -- are not published
yet, and neither is the mcpp release that understands the `[c-abi]`
table. This adds no version entry and no min_mcpp bump for any of that,
because there is no sha256 and no version number to put in one: per
index.toml's own rule, min_mcpp and the CI MCPP_VERSION pin move
together, only once, to a real mcpp release, and raising the floor
early is precisely the "advertising support nobody tested" failure the
2026-08-30 entry in this same file already describes.

What this does add is the structure and a marker of exactly what is
missing, next to where each value will eventually go: a dated note in
index.toml describing the landing order (mcpp releases and enters
xim-pkgindex, then the two packages, then min_mcpp/MCPP_VERSION move
together in the same PR that adds the two versions), and a comment in
each of pkgs/o/openkal-musl.lua and pkgs/o/openkal-llvm-runtime.lua,
next to their version tables, naming the pending version and what it
changes -- without adding that version's xpm entry, since its url would
have no matching sha256 to check it against.

This PR cannot merge before openkal-musl 0.15.0 and openkal-llvm-runtime
0.11.0 are published; this commit is the placeholder for the work that
follows that, not a substitute for it.
… is safe

Follow-up to 8fc4b63, from review: hoisting -include mcpp_zlib_config.h
changes what z_off_t resolves to (off_t instead of a forced long long),
and that flag is package-private -- it does not reach a consumer
compiling zlib.h. Confirmed against tests/examples/zlib's
compile_commands.json: gzlib.c/zutil.c carry -include
mcpp_zlib_config.h, tests/zlib.cpp does not.

So the library and an openkal-Windows consumer compute z_off_t two
different ways (Z_HAVE_UNISTD_H set vs. zconf.h's own unconditional
fallback), and the two only agree because every target this index
measures is LP64, where off_t and long long are both 8 bytes.
tests/openkal/pins.toml and the platforms in this descriptor's own xpm
table confirm there is no ILP32 target measured anywhere today.

Recorded this in the descriptor rather than leaving it implicit, so the
next person who adds an ILP32 target finds the note -- and the existing
rule 3/4 machinery in docs/openkal-compat.md this situation already
matches (a public-header macro that cannot reach the consumer, asserted
at run time via zlibCompileFlags() against sizeof(z_off_t) in
tests/examples/zlib) -- rather than the silent mismatch rule 3 warns
about.

Re-verified: mcpp xpkg parse, lua syntax, and mcpp test for the zlib
member all still pass.
Follow-up to 6382334, from review: the package-level reducer takes
`platform` if any measured target shows it, which is correct behaviour
(a package genuinely posix on Linux and platform-bound only on Windows
does need the platform somewhere) but was not said anywhere a reader of
the badge, rather than the code, would see it.

Made the per-target table the stated authority: the detail-page caption
now adds one sentence, shown only when there is a kind badge for it to
qualify, saying the badge above summarises the strictest measured
target and the table's own `environment` column is what to read for one
target's actual answer. Docs/openkal-compat.md says the same, with the
glfw/tinyhttps example from the plugin's own comment. The plugin's
OPENKAL_KINDS comment and _openkal_kind's docstring say it too, next to
the code that does the reducing.

Verified: py_compile; rebuilt a throwaway copy with synthetic per-target
kind values (posix on both zlib targets, platform on expat's Windows
target only) through the real `xpkgindex generate --offline --strict`
and confirmed the new sentence renders correctly on a mixed package's
detail page, then discarded the copy -- nothing here touches the
checked-in .xpkgindex/openkal-compat.json.
Follow-up to 1844867, from review: the withdrawn `-include unistd.h`
was package-private in exactly the same way the hoisted
`-include mcpp_zlib_config.h` is, so the library and the consumer
already computed z_off_t by two different routes before this change,
and already agreed only because every measured target is LP64. Added
one clause saying so, so the comment does not read as if this PR
introduced the asymmetry it documents.

Re-verified: mcpp xpkg parse, lua syntax, and mcpp test for the zlib
member.
The gate is open: mcpp 2026.9.18.1 (the release that understands
`[c-abi]`) is published and has entered xim-pkgindex. Per index.toml's
own rule, min_mcpp moves together with the CI pin that lints
descriptors against it -- validate.yml's MCPP_VERSION, in this commit.

This also updates a second, separate pin: tests/openkal/pins.toml's
`mcpp` and openkal-compat.yml's own MCPP_VERSION. That pin exists
because tests/openkal/compat.py's `prepare()` opens this repository as
a live index (`[indices] compat = { path = ... }`) to measure against
uncommitted descriptor changes -- so index.toml's min_mcpp gates the
openkal measurement exactly as it gates every other client. Left at
2026.9.17.3 after this raise, every member would fail immediately with
"index requires mcpp >= 2026.9.18.1 but this is mcpp 2026.9.17.3
[E0006]", before a single line of source is read. Confirmed this
failure mode locally before fixing it, rather than reasoning about it.

openkal-musl 0.15.0 and openkal-llvm-runtime 0.11.0 -- the two packages
that will actually declare and consume `[c-abi] presents = "posix"" --
are still not registered anywhere by this commit: no version, no
sha256, nothing invented. tests/openkal/pins.toml's `runtime` field
stays at 0.10.0 for the same reason. This raise is deliberately
separate from that landing step; the two are independent per the
plan's own ordering (mcpp release/xim-pkgindex entry is step 1-2, the
two package releases are step 3, registering them here is step 4).

Verified, not merely asserted, that this raise changes nothing
observable today:
- every descriptor in pkgs/*/*.lua still parses under mcpp 2026.9.18.1
  (`mcpp xpkg parse`, all 231, plus the repo's lua lints)
- a full run of tests/openkal/compat.py against every listed member on
  x86_64-linux-gnu, under 2026.9.18.1, reproduces the checked-in
  baseline exactly: 27 runs / 3 fails, the same three members (expat,
  curl, cmp-module), the same diagnostics character-for-character past
  the local path prefix

Windows was not re-run here: without openkal-musl 0.15.0/
openkal-llvm-runtime 0.11.0, no package declares `[c-abi]` yet, so a
Windows re-run now would only reproduce the existing 15/15 baseline --
no new information, and the expensive (Wine) leg. That baseline's two
members failing inside clang's own headers -- eigen at
mm_malloc.h:43:22 (`__mingw_aligned_malloc` undeclared) and fmtlib.fmt
at intrin.h:12:15 (`'intrin.h' file not found`, clang's own
`#include_next <intrin.h>`) -- are on record here as exactly what
`builtins = "iso"` needs to turn, once openkal-musl 0.15.0 exists to
declare it and the real re-measurement runs.
# Conflicts:
#	.github/workflows/openkal-compat.yml
#	.github/workflows/validate.yml
#	tests/openkal/pins.toml
2026.9.18.2 is the release whose refusals match [c-abi]: an empty
realisation is accepted under GCC, and presents = "posix" is realised on
macOS and on freestanding targets instead of refused. The two packages
that declare the block pin this release in their own CI; the floor must
not leave them refused at 2026.9.18.1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant